fix gcc11 compatibility #4803

aleqs · 2021-11-15T15:10:41Z

Hey,

I recently switched from gcc 10 to gcc 11 (in the project using the c++20 standard) and discovered lightgbm sources no longer compiled for me. This PR fixes the issue for me. No logic is affected, and I checked build compaitbility against gcc 10/11 and clang 12 before submitting.

Alex

ghost · 2021-11-15T15:10:53Z

All CLA requirements met.

jameslamb

Thanks for your interesting in LightGBM!

Can you clarify what you mean by "gcc11 using the c++20 standard"? Did you modify this line that declares that LightGBM follows c++11?

LightGBM/CMakeLists.txt

Line 295 in bfb346c

    
           "${CMAKE_CXX_FLAGS} -std=c++11 -pthread -Wextra -Wall -Wno-ignored-attributes -Wno-unknown-pragmas -Wno-return-type"

And if so, could you help us understanding how C++20 compatibility would help you?

My concern is that right now this project doesn't have tests compiling LightGBM following C++20, so there's even if your PR provides that compatibility today, there's no guarantee that a future PR won't break it. I'd like to understand why supporting a different standard is important to you.

aleqs · 2021-11-15T15:42:10Z

Hey James,

Sorry I think I was being a bit unclear. I made no other changes but the ones I submitted. I referred to the project using lightgbm being under c++20, which as you pointed out should've made no difference. I can see how this can be confusing. I'll edit the issue now to make it clearer.

Alex

aleqs · 2021-11-15T17:31:33Z

I ran a series of tests against the same set of compilers (gcc 10/11, clang 12) in a separate, lightgbm-only environment. All the results are the same: gcc11 is the only one failing before my change, and none of them fail after my change

StrikerRUS · 2021-11-15T19:35:13Z

We are compiling LightGBM with gcc-11 at our CI and have no issues:

LightGBM/.ci/test.sh

Lines 4 to 5 in bfb346c

    
           export CXX=g++-11 
        
           export CC=gcc-11

-- The C compiler identification is GNU 11.2.0
-- The CXX compiler identification is GNU 11.2.0
-- Checking whether C compiler has -isysroot
-- Checking whether C compiler has -isysroot - yes
-- Checking whether C compiler supports OSX deployment target flag
-- Checking whether C compiler supports OSX deployment target flag - yes
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/local/bin/gcc-11 - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Checking whether CXX compiler has -isysroot
-- Checking whether CXX compiler has -isysroot - yes
-- Checking whether CXX compiler supports OSX deployment target flag
-- Checking whether CXX compiler supports OSX deployment target flag - yes
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/local/bin/g++-11 - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Success
-- Using _mm_prefetch
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Success
-- Using _mm_malloc
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/runner/work/LightGBM/LightGBM/build
[  8%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/boosting/boosting.cpp.o
[ 11%] Building CXX object CMakeFiles/lightgbm_capi_objs.dir/src/c_api.cpp.o
[ 11%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/boosting/gbdt.cpp.o
[ 11%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/boosting/gbdt_model_text.cpp.o
[ 14%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/boosting/gbdt_prediction.cpp.o
[ 17%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/boosting/prediction_early_stop.cpp.o
[ 20%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/bin.cpp.o
[ 23%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/config.cpp.o
[ 26%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/config_auto.cpp.o
[ 29%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/dataset.cpp.o
[ 29%] Built target lightgbm_capi_objs
[ 32%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/dataset_loader.cpp.o
[ 35%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/file_io.cpp.o
[ 38%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/json11.cpp.o
[ 41%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/metadata.cpp.o
[ 44%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/parser.cpp.o
[ 47%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/train_share_states.cpp.o
[ 50%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/tree.cpp.o
[ 52%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/metric/dcg_calculator.cpp.o
[ 55%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/metric/metric.cpp.o
[ 58%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/network/ifaddrs_patch.cpp.o
[ 61%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/network/linker_topo.cpp.o
[ 64%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/network/linkers_mpi.cpp.o
[ 67%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/network/linkers_socket.cpp.o
[ 70%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/network/network.cpp.o
[ 73%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/objective/objective_function.cpp.o
[ 76%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/cuda_tree_learner.cpp.o
[ 79%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/data_parallel_tree_learner.cpp.o
[ 82%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/feature_parallel_tree_learner.cpp.o
[ 85%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/gpu_tree_learner.cpp.o
[ 88%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/linear_tree_learner.cpp.o
[ 91%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/serial_tree_learner.cpp.o
[ 94%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/tree_learner.cpp.o
[ 97%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/voting_parallel_tree_learner.cpp.o
[ 97%] Built target lightgbm_objs
[100%] Linking CXX shared library ../lib_lightgbm.so
[100%] Built target _lightgbm

https://github.com/microsoft/LightGBM/runs/4212251986?check_suite_focus=true#step:3:601

Please at least provide the error logs you've observed in your environment.

aleqs · 2021-11-15T20:29:41Z

Sure, please see the log below:

In file included from /home/aleqs/dev/test_project/cmake-build-release-gcc/lightgbm-src/src/io/bin.cpp:16:
/home/aleqs/dev/test_project/cmake-build-release-gcc/lightgbm-src/src/io/dense_bin.hpp:473:28: error: expected unqualified-id before ‘const’
  473 |   DenseBin<VAL_T, IS_4BIT>(const DenseBin<VAL_T, IS_4BIT>& other)
      |                            ^~~~~
/home/aleqs/dev/test_project/cmake-build-release-gcc/lightgbm-src/src/io/dense_bin.hpp:473:28: error: expected ‘)’ before ‘const’
  473 |   DenseBin<VAL_T, IS_4BIT>(const DenseBin<VAL_T, IS_4BIT>& other)
      |                           ~^~~~~
      |                            )
In file included from /home/aleqs/dev/test_project/cmake-build-release-gcc/lightgbm-src/src/io/bin.cpp:17:
/home/aleqs/dev/test_project/cmake-build-release-gcc/lightgbm-src/src/io/multi_val_dense_bin.hpp:220:27: error: expected unqualified-id before ‘const’
  220 |   MultiValDenseBin<VAL_T>(const MultiValDenseBin<VAL_T>& other)
      |                           ^~~~~
/home/aleqs/dev/test_project/cmake-build-release-gcc/lightgbm-src/src/io/multi_val_dense_bin.hpp:220:27: error: expected ‘)’ before ‘const’
  220 |   MultiValDenseBin<VAL_T>(const MultiValDenseBin<VAL_T>& other)
      |                          ~^~~~~
      |                           )
In file included from /home/aleqs/dev/test_project/cmake-build-release-gcc/lightgbm-src/src/io/bin.cpp:18:
/home/aleqs/dev/test_project/cmake-build-release-gcc/lightgbm-src/src/io/multi_val_sparse_bin.hpp:306:7: error: expected unqualified-id before ‘const’
  306 |       const MultiValSparseBin<INDEX_T, VAL_T>& other)
      |       ^~~~~
/home/aleqs/dev/test_project/cmake-build-release-gcc/lightgbm-src/src/io/multi_val_sparse_bin.hpp:305:37: error: expected ‘)’ before ‘const’
  305 |   MultiValSparseBin<INDEX_T, VAL_T>(
      |                                    ~^
      |                                     )
  306 |       const MultiValSparseBin<INDEX_T, VAL_T>& other)
      |       ~~~~~                          
In file included from /home/aleqs/dev/test_project/cmake-build-release-gcc/lightgbm-src/src/io/bin.cpp:19:
/home/aleqs/dev/test_project/cmake-build-release-gcc/lightgbm-src/src/io/sparse_bin.hpp:601:20: error: expected unqualified-id before ‘const’
  601 |   SparseBin<VAL_T>(const SparseBin<VAL_T>& other)
      |                    ^~~~~
/home/aleqs/dev/test_project/cmake-build-release-gcc/lightgbm-src/src/io/sparse_bin.hpp:601:20: error: expected ‘)’ before ‘const’
  601 |   SparseBin<VAL_T>(const SparseBin<VAL_T>& other)
      |                   ~^~~~~
      |                    )

jameslamb · 2021-11-16T15:16:48Z

@aleqs thanks, I think we need more information to understand what's different between your environment and this project's existing CI jobs using gcc 11 (as @StrikerRUS mentioned).

Could you please provide the following?

operating system
exact commands you're using to compile LightGBM
output of cmake --version
output of gcc --version
output of git log -n 1 in your clone of LightGBM`
(if possible) full logs from compilation, not just errors, similar to what @StrikerRUS shared in fix gcc11 compatibility #4803 (comment)

We really appreciate you taking the time to report this and propose this PR, but need to fully understand the issue to be more confident in the fix.

aleqs · 2021-11-17T12:41:33Z

Alright, I'll get back to you with this a bit later, this will take some time on my side

aleqs · 2021-12-20T23:42:13Z

Hey, sorry about the delay. It looks like I won't have time to look into this properly in the foreseeable future, so probably best to mark this issue accordingly and/or close it.

jameslamb · 2021-12-21T03:22:40Z

Thanks @aleqs , we'll close this for now. If you come back to it in the future and are able to provide a reproducible example, we'll be happy to help.

kdarby-cqg · 2023-08-11T21:39:16Z

HI, I am getting the same error using conan (v1) package on v 3.3.5

cmake version 3.26.1

lightgbm/3.3.5: Calling build()
-- The C compiler identification is GNU 12.1.1
-- The CXX compiler identification is GNU 12.1.1
It seems this is a c++ 20 error

https://gcc.gnu.org/pipermail/gcc-patches/2020-May/546183.html

[ 21%] Building CXX object source_subfolder/CMakeFiles/lightgbm.dir/src/io/bin.cpp.o
[ 23%] Building CXX object source_subfolder/CMakeFiles/lightgbm.dir/src/io/config_auto.cpp.o
[ 24%] Building CXX object source_subfolder/CMakeFiles/lightgbm.dir/src/io/config.cpp.o
[ 26%] Building CXX object source_subfolder/CMakeFiles/lightgbm.dir/src/io/dataset.cpp.o
[ 27%] Building CXX object source_subfolder/CMakeFiles/lightgbm.dir/src/io/config_auto.cpp.o
In file included from /home/algo/.conan/data/lightgbm/3.3.5///build/3402b31e727359cf1945dc386665e8a8c324a0f7/source_subfolder/src/io/bin.cpp:16:
/home/algo/.conan/data/lightgbm/3.3.5///build/3402b31e727359cf1945dc386665e8a8c324a0f7/source_subfolder/src/io/dense_bin.hpp:473:28: error: expected unqualified-id before ‘const’
473 | DenseBin<VAL_T, IS_4BIT>(const DenseBin<VAL_T, IS_4BIT>& other)

jameslamb · 2023-08-11T21:51:19Z

Thanks for that! But let's move the discussion off of this old closed PR and to a new issue. I just opened #6033. I'm also working (indirectly) on adding C++20 support in #5981.

If you have any additional details to share that might help with that, please comment on #6033. I'm going to lock discussion here.

fix gcc11/c++20 compatibility

091622f

aleqs requested review from btrotta, chivee, guolinke, hzy46, shiyu1994 and tongwu-sh as code owners November 15, 2021 15:10

jameslamb self-requested a review November 15, 2021 15:17

jameslamb reviewed Nov 15, 2021

View reviewed changes

aleqs changed the title ~~fix gcc11/c++20 compatibility~~ fix gcc11 compatibility Nov 15, 2021

jameslamb added in progress fix awaiting response labels Nov 16, 2021

jameslamb closed this Dec 21, 2021

jameslamb mentioned this pull request Aug 11, 2023

Support C++20 #6033

Closed

microsoft locked as resolved and limited conversation to collaborators Aug 11, 2023

jameslamb removed in progress awaiting response labels Aug 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix gcc11 compatibility #4803

fix gcc11 compatibility #4803

aleqs commented Nov 15, 2021 •

edited

Loading

ghost commented Nov 15, 2021 •

edited by ghost

Loading

jameslamb left a comment

aleqs commented Nov 15, 2021

aleqs commented Nov 15, 2021 •

edited

Loading

StrikerRUS commented Nov 15, 2021

aleqs commented Nov 15, 2021

jameslamb commented Nov 16, 2021

aleqs commented Nov 17, 2021

aleqs commented Dec 20, 2021

jameslamb commented Dec 21, 2021

kdarby-cqg commented Aug 11, 2023

jameslamb commented Aug 11, 2023

fix gcc11 compatibility #4803

fix gcc11 compatibility #4803

Conversation

aleqs commented Nov 15, 2021 • edited Loading

ghost commented Nov 15, 2021 • edited by ghost Loading

jameslamb left a comment

Choose a reason for hiding this comment

aleqs commented Nov 15, 2021

aleqs commented Nov 15, 2021 • edited Loading

StrikerRUS commented Nov 15, 2021

aleqs commented Nov 15, 2021

jameslamb commented Nov 16, 2021

aleqs commented Nov 17, 2021

aleqs commented Dec 20, 2021

jameslamb commented Dec 21, 2021

kdarby-cqg commented Aug 11, 2023

jameslamb commented Aug 11, 2023

aleqs commented Nov 15, 2021 •

edited

Loading

ghost commented Nov 15, 2021 •

edited by ghost

Loading

aleqs commented Nov 15, 2021 •

edited

Loading